%load_ext autoreload
%autoreload 2
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
import pandas as pd
import plotly.io as pio
from idea.profile.profile import calculate_profile
from idea.validation.validation import validate_roadwork
from util.visualization import create_multiplot_heatmap, create_plot_by_segment
pd.options.plotting.backend = "plotly"
pio.renderers.default = "notebook"
Example Validations¶
This repository contains three example cases:
Example 1: There is always the maximum amount of FCD coverage during the profile period. Even during the roadworks, coverage does not decrease. → Status:
openExample 2: There is always the maximum amount of FCD coverage during the profile period. During the roadworks, there is no coverage, but occasionally a vehicle is detected. → Status:
closedduring periods with less than usual coverageExample 3: There is consistently very low FCD coverage during the profile period. During the roadworks, coverage drops to none, with occasional vehicle detection. There is one short period where coverage increases, and the algorithm marks it as
open. → Status:closed, with one shortopenperiod due to increased coverage
# Choose the test you want to run (1 till 3 available)
EXAMPLE_FOLDER = "example2"
# Retrieve fcd during profile period.
df_minute = pd.read_csv(rf"resources/{EXAMPLE_FOLDER}/fcd_profile_input.csv", sep=";", index_col=0)
df_minute.index = pd.to_datetime(df_minute.index, utc=True)
start = pd.Timestamp("2024-01-01 00:00:00", tz="UTC")
end = pd.Timestamp("2025-01-01 00:00:00", tz="UTC")
profile = calculate_profile(df_minute, start, end)
# Retrieve fcd during roadwork.
validation_start = pd.Timestamp("2025-03-15 12:00:00", tz="UTC")
validation_end = pd.Timestamp("2025-03-15 15:00:00", tz="UTC")
fcd_during_roadwork = pd.read_csv(
rf"resources/{EXAMPLE_FOLDER}/fcd_during_roadwork.csv", sep=";", index_col=0
)
fcd_during_roadwork.index = pd.to_datetime(fcd_during_roadwork.index, utc=True)
validated_roadwork = validate_roadwork(fcd_during_roadwork, profile)
validated_roadwork
| time | fcd | consecutive_zeros | consecutive_low | day_of_week | hour_of_day | max_consecutive_zeros_q95 | max_consecutive_zeros_or_ones_q95 | fcd_mean_median | running_mean | segment_closure_status | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2024-01-05 00:00:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.500000 | undetermined |
| 1 | 2024-01-05 00:01:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.166667 | undetermined |
| 2 | 2024-01-05 00:02:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.055556 | open |
| 3 | 2024-01-05 00:03:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.018519 | open |
| 4 | 2024-01-05 00:04:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.006173 | open |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14395 | 2024-01-14 23:55:00+00:00 | 10 | 0 | 0 | Sunday | 23 | 0.0 | 0.0 | 10.0 | 0.000000 | open |
| 14396 | 2024-01-14 23:56:00+00:00 | 10 | 0 | 0 | Sunday | 23 | 0.0 | 0.0 | 10.0 | 0.000000 | open |
| 14397 | 2024-01-14 23:57:00+00:00 | 10 | 0 | 0 | Sunday | 23 | 0.0 | 0.0 | 10.0 | 0.000000 | open |
| 14398 | 2024-01-14 23:58:00+00:00 | 10 | 0 | 0 | Sunday | 23 | 0.0 | 0.0 | 10.0 | 0.000000 | open |
| 14399 | 2024-01-14 23:59:00+00:00 | 10 | 0 | 0 | Sunday | 23 | 0.0 | 0.0 | 10.0 | 0.000000 | open |
14400 rows × 11 columns
# Add order column (needed for plot function)
validated_roadwork["order"] = 1
validated_roadwork.head()
| time | fcd | consecutive_zeros | consecutive_low | day_of_week | hour_of_day | max_consecutive_zeros_q95 | max_consecutive_zeros_or_ones_q95 | fcd_mean_median | running_mean | segment_closure_status | order | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2024-01-05 00:00:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.500000 | undetermined | 1 |
| 1 | 2024-01-05 00:01:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.166667 | undetermined | 1 |
| 2 | 2024-01-05 00:02:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.055556 | open | 1 |
| 3 | 2024-01-05 00:03:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.018519 | open | 1 |
| 4 | 2024-01-05 00:04:00+00:00 | 10 | 0 | 0 | Friday | 0 | 0.0 | 0.0 | 10.0 | 0.006173 | open | 1 |
Visualization of the validation values for a single segment.¶
fig = create_multiplot_heatmap(
validated_roadwork,
[
"fcd",
"running_mean",
"segment_closure_status",
"fcd_mean_median",
"max_consecutive_zeros_q95",
"max_consecutive_zeros_or_ones_q95",
],
)
fig.show()
Visualization of the running mean for a single segment¶
fig = create_plot_by_segment(validated_roadwork)
fig.show()